Automated image analysis to assess reproductive potential of human oocytes and pronuclear embryos

ABSTRACT

A computer system automatically converts a set of training images of cells (e.g., oocytes or pronuclear embryos) and related outcome metadata into a description document by extracting features (e.g., cytoplasm features) from the pixel values of the training images that describe the cells and associating the extracted features with the outcome metadata. Based on the description document, the system automatically computes a decision model that can be used to predict outcomes of new cells. To predict outcomes of new cells, a computer system automatically extracts features from images that describe the new cells and predicts one or more outcomes by applying the decision model. The features extracted from the images that describe the new cells correspond to features selected for inclusion in the decision model, and are calculated in the same way as the corresponding features extracted from the training images.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/288,379, filed Jan. 28, 2016, the disclosure of which is herebyincorporated by reference herein.

BACKGROUND

Conventional assessment of embryo quality is performed through static,daily measurements by observers using traditional bright fieldexamination of embryos during 5 days of development. Though a variety ofscoring systems can be used to translate these observations intosemi-quantifiable scores, selection of the embryo or embryos most likelyto implant remains a qualitative exercise. This process is labor andcost intensive, requires considerable training and skill, and may beimpacted by factors common to any repetitive technique of observationsuch as fatigue and inter- and intra-observer variability.

Conventionally, oocytes are retrieved and immediately sorted based onlyon maturity or immaturity, with the former being inseminated.Performance of these oocytes awaits an expensive and time consumingroutine of culturing over 5 days.

Manual oocyte scoring may provide useful clinical information. However,manual assessment of oocytes and embryos remains standard of care andhas not changed significantly since inception of human embryologytechniques. Thus, there remains a need for better tools to assess thereproductive potential of oocytes and pronuclear embryos to identifythose with a high likelihood of developing into blastocysts andultimately a live birth.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features ofthe claimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

In one aspect, a computer system automatically converts a set oftraining images of cells (e.g., oocytes or pronuclear embryos) andrelated outcome metadata into a description document by extractingfeatures from the pixel values of the training images and associatingthe extracted features with the outcome metadata. Then, based on thedescription document, the computer system automatically computes adecision model that can be used to predict outcomes of new cells. Thetraining images are obtained using bright-field microscopy.

The extracted features may include a boundary of a cytoplasm. Thecomputer system may detect such boundaries in the training images byapplying an edge detection algorithm to the images to detect edges andapplying a circle detection algorithm to the detected edges to detect anapproximate cytoplasm boundary. The computer system may convert colorimages to grayscale images prior to application of the edge detectionalgorithm. After the approximate cytoplasm boundary is detected, thecomputer system may create an annular mask image from the approximatecytoplasm boundary and perform image segmentation on the original sourceimage to identify a connected region as the cytoplasm.

Additional extracted features may include one or more of the following:total area of the cytoplasm; aspect ratio of the cytoplasm; convexity ofthe boundary of the cytoplasm; average image intensity of the cytoplasm;standard deviation of the image intensity of the cytoplasm; smoothnessof the cytoplasm; subsampled image intensity or smoothness for gridareas within the cytoplasm; cytoplasmic texture or texture distribution;density; and clustering. The average image intensity of the cytoplasmmay be calculated by averaging the corresponding pixel values of theidentified cytoplasm in a grayscale image.

For oocytes or pronuclear embryos, additional extracted features mayinclude one or more of the following: a boundary of a polar body,measured manually or with image analysis using an edge detectionalgorithm; an indication as to whether the polar body exists;identification of inner and outer edges of a zona pellucida;identification of perivitelline space; a measurement of smoothness ofthe edges of the zona pellucida; alignment of principal axes of thecytoplasm and the zona; total area of the polar body, the zonapellucida, or the perivitelline space; aspect ratio of the polar body,the zona pellucida, or the perivitelline space; convexity of a boundaryof the polar body, the zona pellucida, or the perivitelline space;average image intensity of the polar body, the zona pellucida, or theperivitelline space; standard deviation of the image intensity of thepolar body, the zona pellucida, or the perivitelline space; smoothnessof the polar body, the zona pellucida, or the perivitelline space; andsubsampled image intensity or smoothness for grid areas within the polarbody, the zona pellucida, or the perivitelline space.

For pronuclear embryos, additional extracted features may include one ormore of the following: identification of the boundaries of thepronuclei, measured manually or with image analysis using an edgedetection algorithm; area of the pronuclei, measured individually and/orrelative to each other; aspect ratio of the pronuclei, measuredindividually and/or relative to each other; convexity of a boundary ofone or more of the pronuclei; average image intensity of one or more ofthe pronuclei; standard deviation of the image intensity of one or moreof the pronuclei; smoothness of one or more of the pronuclei; andsubsampled image intensity or smoothness for grid areas within one ormore of the pronuclei.

In another aspect, a computer system automatically extracts featuresfrom new cell images (which, like the training images, also may beobtained using bright-field microscopy) that describe the new cells andpredicts one or more outcomes for the new cells by applying the decisionmodel to the features extracted from the new cell images. The featuresextracted from the new cell images correspond to features of thetraining images selected for inclusion in the decision model, and may becalculated for in the same way. The outcomes may include one or more ofquality or reproductive potential for a new oocyte, whetherabnormalities in chromosome number are detected later in development,whether genetic disorders are detected later in development, andabnormal cell growth outcomes. The quality or reproductive potential fora new oocyte may be expressed as one or more of: a probability that theoocyte will fertilize when exposed to human sperm, a probability that aresulting embryo will reach a given stage of development, or aprobability that the resulting embryo will result in a live birth whentransferred to a uterus.

These functions may be implemented as an add-on to an existing imageanalysis system or as a separate system. The system may be integratedwith a microscopy system for obtaining cell images.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisinvention will become more readily appreciated as the same become betterunderstood by reference to the following detailed description, whentaken in conjunction with the accompanying drawings, wherein:

FIG. 1A is an image of a mature oocyte that may be analyzed by anautomated image-based cell analysis system according to at least oneembodiment of the present disclosure;

FIG. 1B is an image of abnormal oocytes that may be analyzed by anautomated image-based cell analysis system according to at least oneembodiment of the present disclosure;

FIG. 2 is a flow diagram illustrating work flow in an automatedimage-based cell analysis system according to at least one embodiment ofthe present disclosure;

FIG. 3 is a flow diagram illustrating an algorithm for identification ofa boundary of a cytoplasm using image analysis techniques according toat least one embodiment of the present disclosure;

FIG. 4 is a flow diagram illustrating an algorithm for identification ofinner and outer edges of a zona pellucida of an oocyte or pronuclearembryo according to at least one embodiment of the present disclosure;

FIG. 5 is an image of a pronuclear embryo that may be analyzed by anautomated image-based cell analysis system according to at least oneembodiment of the present disclosure; and

FIG. 6 is a block diagram that illustrates aspects of an illustrativecomputing device appropriate for use in accordance with embodiments ofthe present disclosure.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth inorder to provide a thorough understanding of illustrative embodiments ofthe present disclosure. It will be apparent to one skilled in the art,however, that many embodiments of the present disclosure may bepracticed without some or all of the specific details. In someinstances, well-known process steps have not been described in detail inorder not to unnecessarily obscure various aspects of the presentdisclosure. Further, it will be appreciated that embodiments of thepresent disclosure may employ any combination of features describedherein. The illustrative examples provided herein are not intended to beexhaustive or to limit the claimed subject matter to the precise formsdisclosed.

The present disclosure is directed to computerized image analysis,content-based image retrieval and cellular feature extraction of cellssuch as oocytes and pronuclear embryos, and their various intracellularcomponents, and predicting outcomes related to cell quality, cellgrowth, or development (e.g., developmental changes that occur incomparison between oocytes and pronuclear embryos). The presentdisclosure includes an algorithm for the study of attributes of oocytesand pronuclear embryos that can be used in an automated analyticsoftware tool to identify oocytes and pronuclear embryos with a highlikelihood of implanting, among other possible outcomes related toreproductive potential. The present disclosure describes technology withunique clinical benefits distinct from anything commercially available.Disclosed embodiments are particularly significant in their ability toadapt to new cell science; as new outcomes are studied and linked toparticular cell features, disclosed embodiments offer the ability toautomatically extract those features from training images and associatewith them with outcomes, generate decision models, and apply thosedecision models to new cell images in order to predict outcomes for thenew cells.

Software systems for the interpretation of images are recent additionsto the catalogue of programs with clinical application. Systems are nowavailable for pattern and texture extraction in defined regions ofinterest. Extracted features can be added to catalogues to createarchives of normal and abnormal images for comparisons and ultimatelyimproved decision making and outcomes as well as possible costreductions and improvements in work flow within labs. Video imageanalysis of embryos using dark field assessments has been described. Nosoftware, however, has been tested that evaluates attributes of oocytesor pronuclear embryos in standard bright field formats using thetechniques described herein.

The present disclosure describes aspects of an automated analysis systemthat evaluates cells, such as oocytes and pronuclear embryos, in a newway while still being compatible with hardware systems using traditionalbright field microscopy. Other computer image analysis systems usetime-lapsed dark-field systems, and require separate monitoring hardwareand additional personnel for maintenance and monitoring. In addition,data analyzing outcomes with such systems are of variable quality andhave not consistently demonstrated any improvement in outcomes.

As suggested above, in contrast to existing systems, embodiments of thepresent disclosure have several potential marketplace advantages. As onepossible advantage, existing hardware (e.g., in embryology labs) can beleveraged for additional benefit using embodiments of the presentdisclosure. For example, existing microscopy systems present in mostlabs may be used, with no additional hardware needed. As anotherpossible advantage, existing image analysis software can be extended byincorporating embodiments of the present disclosure into an existingimage analysis software system or into an integrated system ofmicroscopy hardware an image analysis software.

The archive of images and outcomes also can be expanded to enhanceprecision and reliability of the algorithm. Image training can beperformed on training images obtained from multiple sources (e.g.,clinics, research institutions) stored in a common data repository.Using artificial intelligence, machine self-training can provideimprovements over time thus leading to significant improvements inpredicting outcomes with continued use. Embodiments described herein canbe applied, for example, to both oocyte cryopreservation and embryodevelopment to predict reproductive potential in oocytes prior tocryopreservation or after insemination during embryo development.

Further, embodiments described herein can provide useful analysis of asingle cell, potentially offering greater ease in image acquisition andanalysis, as well as improved clinical decision making as it relates tocell quality or development, e.g., prior to oocyte freezing forfertility preservation or oocyte donation and improved selection ofwhich oocyte to inseminate and which pronuclear embryo to continue inculture. Embodiments described herein are useful in studying singlecells, particularly in the context of reproductive medicine. Afterinsemination and culturing, as the embryo expands in cell number (e.g.,beyond 2 days of observation), the image field becomes more crowded, itbecomes difficult to isolate individual cells, and the analytics becomemore complicated. As single cells, oocyte and pronuclear embryos provideexcellent opportunities to extract information using computerized imageanalysis.

The present disclosure describes a quality model learned from existingimages and related metadata, such as descriptions of outcomes related tocell quality, cell growth, and development. In the context ofreproductive medicine, such outcomes include blastocyst formation,ploidy determinations, and clinical pregnancies. Reproductive potentialcan be assessed for, e.g., oocytes after freezing and pronuclearembryos. For example, as new forms of data become available andregulatory approvals are obtained for additional study (which mayprovide additional descriptions of outcomes, or new classes ofoutcomes), the dataset of existing images, metadata, and outcomes can beextended and the quality models can be improved.

The quality or reproductive potential of an oocyte or pronuclear embryocan be expressed in several different ways, such as the probability thatthe oocyte will fertilize when exposed to human sperm, the probabilitythat the resulting embryo will reach a given stage of development (e.g.,the blastocyst stage), or the probability that the embryo will result ina live birth when transferred to a uterus. In turn, probability can beexpressed as a percentage chance, or as a high/medium/low chance, ofreaching a given state of development. In some circumstances, there mayalso be some probability that genetic information (e.g., the geneticcomplement) or abnormal cell growth outcomes can be predicted, usingtechniques described in the present disclosure. For example, as newinformation becomes available linking genetic information or abnormalcell growth with cell features that can be identified in images usingthe image analysis techniques described herein, these techniques cancorrespondingly be used to deduce genetic information or abnormal cellgrowth outcomes from such images. In one possible scenario, instead ofan outcome indicating whether an oocyte was successfully fertilized orhow embryo development proceeded, an outcome may be whetherabnormalities in chromosome number or genetic disorders are detectedlater on in development.

Additionally, aspects of the present disclosure provide the ability togauge oocyte quality to guide decision making regarding oocytecryopreservation. Oocyte preservation may be performed, for example, inan elective oocyte freezing program, in which it may be helpful todetermine how many oocytes are to be frozen to maximize futurefertility, or to determine a point of diminishing returns beyond whichadditional oocyte retrievals do not significantly increase theprobability of future fertility. Thus, aspects of the present disclosuremay be used to avoid performing too many or too few oocyte retrievals.Oocyte preservation also may be performed in an oocyte donation program,where the ability to prospectively assess oocyte potential will guidedecision making about how many oocytes to batch for a given donation orwhether to discard them and forego fertilization.

Aspects of the present disclosure improve on the current practice ofusing a single determination of whether an oocyte is mature or immatureand freezing 6 to 8 oocytes per batch. Aspects of the present disclosuremay be particularly useful in scenarios where the number of oocytesinseminated is limited, and the ability to identify an oocyte with thegreatest reproductive potential becomes more important.

FIG. 1A is an image of a mature oocyte, with the zona pellucida (or“zona”) and polar body indicated. In an illustrative embodiment,attributes of the cytoplasm, zona, and polar body can be automaticallyidentified and analyzed, as described in detail below.

In an illustrative embodiment, the overall method is composed of twostages, each of which is composed of two phases. Stage one, the modelconstruction stage, involves constructing a decision model (e.g., amodel of quality for human oocytes or pronuclear embryos) from trainingimages of cells (e.g., of oocytes or pronuclear embryos for whichparticular outcomes are known). Stage two, the employment stage,involves employing this model to predict outcomes for new cells beinganalyzed (e.g., in a clinical setting). For example, in the context ofreproductive medicine, the employment stage may be used to determine thequality of an individual embryo.

The method may be performed by an automated image-based cell analysissystem, as described herein, implemented in a computer system comprisingone or more computing devices. At a high level the automated image-basedcell analysis system may include functionality for performing tasksdescribed with reference to FIG. 2. The automated image-based cellanalysis system may include multiple modules or engines to carry outdifferent groups of tasks associated with different stages or phases,such as a model construction engine and a model employment engine, asdescribed in detail below. Such modules or engines may be implemented inthe same computing device, or in different computing devices. In turn,these sections or engines may include multiple submodules to carry outmore specific tasks within the groups of tasks, such as particular typesof feature extraction within the model construction engine.

Phase 1, the first phase of the model construction stage, is the featureextraction phase. In the example shown in FIG. 2, in phase 1 theautomated image-based cell analysis system obtains a set of trainingimages and related metadata from a data store 210 and converts them(e.g., using a model construction engine) into a description document.Like other digital images, the digital images that form the training setcomprise pixels arranged in rows and columns. Each pixel is defined by aset of one or more sample values, with the number, meaning, and valuerange of the respective samples being determined by the format, colorspace, and bit depth (number of bits per channel) of the images. It willbe understood that the techniques described herein can be adapted foranalysis of digital images of various formats, resolutions, colorspaces, bit depths, etc. For example, although some techniques aredescribed herein as being performed on grayscale images at apredetermined resolution, various techniques for upscaling ordownscaling image resolution, converting color images to grayscale, andthe like, can be employed in combination with techniques describedherein.

The feature extraction phase proceeds by extracting features 212 fromthe training images that describe the cells (e.g., oocytes or pronuclearembryos) in the images numerically, as described in detail below. In thecontext of reproductive medicine, this may include extraction offeatures such as the area measurement of the cytoplasm of each oocyte,or cytoplasmic or nuclear features of each pronuclear embryo in thetraining images. Although the features described herein focus ontwo-dimensional features such as two-dimensional boundaries, area withinsuch boundaries, etc., it will be understood that such features may beextended to some three-dimensional features, such as volume, to theextent that such features can be accurately estimated based oninformation obtained from two-dimensional images.

The extracted features are then associated with outcomes in thedescription document 214. Further details of the features that can beincluded in a description document are provided below. (The term“document” is used herein to refer to a collection of the describedinformation in any format suitable for further processing as describedherein, and need not refer to a traditional document designed for humanreadability.) The data store may be updated as needed, e.g., to provideadditional training images and related metadata.

In an illustrative embodiment, one required item in the descriptiondocument is the description of one or more outcomes of what happenedwhen the oocytes in the respective training images were fertilized. Theoutcomes associated with training images, also referred to herein as“Feature 1” of the respective training images, are not a function of theimage data, but are metadata supplied along with the images. Themetadata may include information related to developmental outcomes. Withregard to a training image of an oocyte, for example, such metadata mayinclude information about whether the oocyte fertilized and how embryodevelopment proceeded.

In at least one embodiment, one or more of the following features 2-18can be extracted from an oocyte image to create the description documentin phase 1, as described below. It will be understood that extraction ofsome features described below (e.g., cytoplasm features) may be usefulfor many types of cells. For example, the pronuclear embryo is still asingle cell (see FIG. 5), and its features also can be analyzed asdescribed below:

-   -   2. Identification of the boundary of the cytoplasm, measured        manually or with image analysis using an edge detection        algorithm. An illustrative algorithm 300 is illustrated in FIG.        3:        -   (a) At step 310, convert image from color to grayscale (if            needed).        -   (b) At step 320, apply an edge detection algorithm (e.g.,            Canny Threshold) to the grayscale image to detect edges.        -   (c) At step 330, apply a circle detection algorithm (e.g.,            Hough Circle detection) to find an approximate cytoplasm            boundary. In at least one embodiment, the circle detection            algorithm may use known quantities of image resolution and            magnification to control for size of circles to search from,            and Gaussian blur on Canny-detected edges to further reduce            number of detected circles, if needed, until only a single            circle remains.        -   If further processing steps that rely on the detected            cytoplasm are to be performed, the following steps may be            helpful:        -   (d) At step 340, create an annular mask image from the            circle, with inner and outer annulus radii as ratios of            circle radius.        -   (e) At step 350, perform image segmentation on the original            source image to identify a connected region as the            cytoplasm. For example, a GrabCut method can be used, with            the resulting connected region being the cytoplasm, and the            boundary of the connected region being the boundary of the            cytoplasm. The annular mask can be used to restrict the            search.        -   A mask image may be created containing only pixels from the            connected region. This image simplifies computing further            cytoplasm features.    -   3. Total area of the cytoplasm in the image, defined by the        portion of the image inside the boundary identified in        feature 2. (The volume of the cytoplasm can also be estimated        based on this information.)    -   4. Aspect ratio of the cytoplasm, being the ratio of the        shortest distance across the cytoplasm to the longest.    -   5. Convexity of the cytoplasm boundary. This feature may relate        to comparative changes in the cytoplasm transitioning from        oocyte to pronuclear embryo. As an example, convexity may be        calculated as follows:        -   (a) Compute the convex hull of the mask image generated in            feature 2, step (e), above.        -   (b) Calculate the ratio of the area of the mask and area of            the convex hull.    -   6. Average image intensity of the cytoplasm. For example,        average intensity may be calculated by averaging the        corresponding pixel values of the identified cytoplasm in a        grayscale image (in which the value of each pixel is a single        sample), or by averaging a particular value for corresponding        pixels (such as brightness) when represented in a color space        such as HSB (Hue-Saturation-Brightness).    -   7. Standard deviation of the image intensity of the cytoplasm.    -   8. Smoothness of the cytoplasm, measured by applying a Gaussian        filter to the cytoplasm image, varying the size of the Gaussian        filter and recording how that affects the number of features        that can be detected in the cytoplasm by, e.g., Canny edge        detector, Harris corner detector, or another process which        detects features which vary by image smoothness. As the Gaussian        smoothing is increased the number of features decreases until no        features are detected. The system can track how fast the number        of features decreases and how much smoothing is needed to        eliminate features entirely, which provides information about        the smoothness of the cytoplasm.    -   9. Subsampled examination of the cytoplasm, where the cytoplasm        area is divided by a grid, and each of the grid areas is        subjected to techniques 6, 7 and 8. The results of these        procedures can be characterized by:        -   a. variation of the outcomes for the individual grid areas,            or        -   b. counts generated when given thresholds are exceeded for a            grid area.    -   10. Identification of the boundary of the polar body, measured        manually or with image analysis using an edge detection        algorithm.    -   11. Indication based on feature 10, above, as to whether the        polar body exists for the oocyte or pronuclear embryo.    -   12. For the polar body, features analogous to features 3-9,        above.    -   13. Identification of the inner and outer edges of the zona        pellucida of the oocyte or pronuclear embryo. The inner boundary        may be coincident with the boundary of the cytoplasm, or gaps        may exist between them (perivitelline space). An illustrative        algorithm 400 is illustrated in FIG. 4:        -   (a) At step 410, to identify the outer zona boundary, mask            out the region of the image that has been identified as            containing the boundary of the cytoplasm in feature 2,            above.        -   (b) At step 420, to identify the inner zona boundary, mask            out parts of the image inside the cytoplasm and outside the            outer zona boundary, and then identify the inner zona            boundary by applying edge detection and circle detection            algorithms as in feature 2.        -   In addition, image sharpening can be performed before the            edge detection steps so that the zona edges are more readily            apparent.    -   14. For the zona pellucida, features analogous to features 3-9,        above.    -   15. A measurement of the smoothness of the boundaries of the        zona, describing the size and number of features found on the        edges.    -   16. Alignment of the principal axes of the cytoplasm and the        zona. For example, the system may calculate axes by finding a        minimal bounding rectangle around the mask of the cytoplasm and        the zona, with alignment of the sides of the rectangle defining        the principal axes. As another example, the system may determine        alignment by estimating the oocyte perimeter from an interior        reference point (e.g., centroid) and iteratively region-growing        no further than the oocyte edge, passing coordinates through        this centroid along its major and minor axes to the oocyte edge,        and measuring coordinate lengths within the estimated oocyte        region.    -   17. Area of the perivitelline space between the cytoplasm and        the zona. This is the area within the inner zona which is not        also within the cytoplasm.    -   18. For the perivitelline space, features analogous to features        3-9, above.

An illustrative description document for 6 images with outcomes and 5features associated with those images is shown in Table 1, below.

TABLE 1 Illustrative description document Cytoplasm Intensity Intensitystd Canny Harris Image Outcome size ratio mean dev 1.000000 13.000000 11 0.126841 0.478617 0.0623425 0.187357 0.0688431 2 0 0.108994 0.4657570.0623978 0.198311 0.0519201 3 0 0.117361 0.455848 0.0678655 0.1973250.0439613 4 1 0.0966727 0.514367 0.0622328 0.0312051 0.00810862 5 10.108249 0.513011 0.0877929 0.164327 0.0360605 6 0 0.117361 0.4720420.0797936 0.179902 0.0625631

Other features that may be extracted with image analysis techniquesinclude cytoplasmic texture features, texture distribution, densities,and clustering.

An illustrative image of a pronuclear embryo is shown in FIG. 5. Forpronuclear embryos, in addition to the applicable features describedabove, any of the additional features 19-23 described below can beextracted for the pronuclei in the center of the cytoplasm, as follows:

-   -   19. Identification of the boundaries of the pronuclei, measured        manually or with image analysis using an edge detection        algorithm.    -   20. The area or volume of the pronuclei, measured individually        and relative to each other.    -   21. The aspect ratio of the pronuclei measured individually and        relative to each other.    -   22. For the pronuclei, features analogous to features 5-9,        above, and comparisons of those features between the two        pronuclei.    -   23. For each of the features that are common to pronuclei and        oocyte analysis, generate a set of comparison features by        subtraction or division.

The features described herein may be used to detect abnormalities in,for example, texture, inclusion bodies, and shape or size, such as theabnormalities depicted in the oocytes shown in FIG. 1B.

In addition to the image analysis tools described above, other tools,such as colorization and color filters, also may be used for featureextraction. For example, to the extent that a particular color filter isuseful for detecting a particular feature, the color filter can beapplied to a color image of the cell before proceeding with furtheranalysis of the image. This process may be performed, for example, on acolor source image prior to conversion of the color source image tograyscale for detection of other features.

As noted above, it is not required for all of the features describedabove to be extracted in every case, or for all features to be extractedautomatically. In some cases, automatic analysis can be performed onmanually measured features (e.g., features measured with the help of acustomized graphical user interface) or on combinations of manuallymeasured features and automatically extracted features.

Referring again to FIG. 2, the second phase of the model constructionstage (phase 2), is the learning or training phase. Phase 2 determineswhich features extracted in phase 1 materially affect predictive abilityin phase 4. Features selected for inclusion in the decision model mayinclude, for example, features having a numerical predictive value thatexceeds a threshold value. Features not selected for inclusion in thedecision model, which do not have significant predictive value, can beexcluded in phases 3 and 4, which are described in detail below. Inphase 2, the model construction engine consumes or reads (e.g., usingthe model construction engine) the descriptions (e.g., numericdescriptions) in the description document and computes a mathematicaloptimization to determine which of the features are significant indetermining quality of outcomes. Thus, the output of phase 1 is theinput for phase 2. The output of phase 2 is a model (see “generatedecision model” block 220 shown in FIG. 2) that can be used to predictoutcomes. For example, in the field of reproductive medicine, suchpredicted outcomes may be used to determine quality or reproductivepotential given the numeric features for a given oocyte or pronuclearembryo. This can be referred to as constructing a model with labeledtraining data. In at least one embodiment, the image analysis andattributes that the software detects are correlated with outcome andweighted according to significance.

In at least one embodiment, a portion of the training data is held outto use for validating the model, and training is done on the remainingtraining data using cross-validation to ensure the stability of themodel. The model can be constructed using logistic regression, decisiontrees, or some other method that consumes labeled training data.Sufficient training data is required to achieve a significant resultfrom training. When cross-validation implies that the model ispredictive, the held-out portion of the training data is used tovalidate model quality. Training in phase 2 also may be performed usingunsupervised learning such as k-means or neural networks to segment thetraining data into classes. After classes have been identified, thepredominant label from the samples in each class is assigned to eachclass. The learning phase also may be performed using other machinelearning methods.

As noted above, the output of phase 2 is a decision model which can beused to predict outcomes, such as cell quality or reproductivepotential. As further noted above, the quality or reproductive potentialof an oocyte or pronuclear embryo can be expressed in several differentways, such as the probability that the embryo will be chromosomallynormal or euploid, or abnormal or aneuploid, the probability that theoocyte will fertilize when exposed to human sperm, the probability thatthe resulting embryo will reach a given stage of development (e.g., theblastocyst stage), or the probability the embryo will result in a livebirth when transferred to a uterus. In turn, probability can beexpressed as a percentage chance, or as a high/medium/low chance, ofreaching a given state of development.

In phase 3, the first phase of the employment stage, the automatedimage-based cell analysis system performs (e.g., using the modelemployment engine) feature extraction on a new image 230 (e.g., an imageof a single oocyte or pronuclear embryo). In this phase, the automatedimage-based cell analysis system generates features 232 for the newimage corresponding to one or more features that were generated for thetraining images in phase 1, other than feature 1 (outcome), which is notyet known. It is not necessary for all of the features described inphase 1 to be extracted for new images in phase 3. For example, featuresthat have been determined in phase 2 not to have predictive value neednot be extracted in phase 3.

Although automatic feature extraction has many benefits, as describedherein, it is also possible to use the techniques described herein onone or more manually extracted features. For accurate modeling, featuresthat are extracted manually in phase 1 can also be extracted manually inphase 3.

To ensure that values for the features described herein can be comparedbetween images, for both learning and employment phases, a rule that allimages be of the same resolution and magnification can be enforced. Tocomply with this rule, the original images do not necessarily need to beof the same resolution and magnification. Instead, images can be scaledand cropped as needed, either manually or automatically.

In phase 4, the automated image-based cell analysis system predicts(e.g., using the model employment engine) one or more outcomes 242 forthe cell (e.g., oocyte or pronuclear embryo) in the new image byapplying the decision model 240 generated in phase 2 to the featuresextracted in phase 3. For example, a score between 0 and 1 can beassigned, with a greater score indicating a better predicted outcome.The predicted outcome adds a significant benefit compared to, forexample, older methods of selecting oocytes for insemination basedsolely on maturity. A threshold score can be set, below which the oocyteregardless of maturity may be discarded as unlikely if not impossible toyield a blastocyst and live birth. Depending on the design of thedecision model, the features may be weighted together in somecombination, or used as decision points in a decision tree applicationto determine the likely outcomes for the cell (e.g., oocyte orpronuclear embryo) for one or more of the scenarios provided as outcomesin feature 1 of the training set.

Cells can be analyzed separately, or in combination. For example, thequality of oocytes and pronuclear embryos can be analyzed individuallyand then combined to form an overall quality score.

As described above, although many features may be extracted, somesubsets may be more predictive than others. The system can use varioustypes of predictive analytics to determine which features orcombinations of features are more predictive. In an illustrativeembodiment, an automated image-based cell analysis system identifies 5cellular attributes of oocytes. These attributes were selected aspossibly predictive of the likelihood reproductive potential of theoocyte and of blastocyst formation and implantation. A numerical scorebetween 0.0 (lowest) and 1.0 (highest) was calculated to estimate theselikelihoods. In this illustrative embodiment, the algorithm employsStochastic Gradient Descent implemented in the Python programminglanguage through the SciPy extension. Feature extraction uses a varietyof additional techniques in C++, including a Hough Circle detector and aHarris Corner detector. Main outcome measurements include calculation ofan Oocyte Score (OS) to identify oocytes with a high reproductivepotential and likelihood of blastocyst formation and to compare the OS(Group 1) to selection using standard morphology (Group 2) and to acomputerized video monitoring system (Group 3).

In at least one embodiment, an automated image-based cell analysissystem can receive software updates over a network, to provide updatesto the analysis system, the training data, or other data.

Illustrative Devices and Operating Environments

Unless otherwise specified in the context of specific examples,described techniques and tools may be implemented by any suitablecomputing device or set of devices.

In any of the described examples, an engine (e.g., a software engineworking in combination with computer hardware and microscopy systems)may be used to perform actions described herein. An engine includeslogic (e.g., in the form of computer program code) configured to causeone or more computing devices to perform actions described herein asbeing associated with the engine. For example, a computing device can bespecifically programmed to perform the actions by having installedtherein a tangible computer-readable medium having computer-executableinstructions stored thereon that, when executed by one or moreprocessors of the computing device, cause the computing device toperform the actions. The particular engines described herein areincluded for ease of discussion, but many alternatives are possible. Forexample, actions described herein as associated with two or more engineson multiple devices may be performed by a single engine. As anotherexample, actions described herein as associated with a single engine maybe performed by two or more engines on the same device or on multipledevices.

In any of the described examples, a data store contains data asdescribed herein and may be hosted, for example, by a databasemanagement system (DBMS) to allow a high level of data throughputbetween the data store and other components of a described system. TheDBMS may also allow the data store to be reliably backed up and tomaintain a high level of availability. For example, a data store may beaccessed by other system components via a network, such as a privatenetwork in the vicinity of the system, a secured transmission channelover the public Internet, a combination of private and public networks,and the like. Instead of or in addition to a DBMS, a data store mayinclude structured data stored as files in a traditional file system.Data stores may reside on computing devices that are part of or separatefrom components of systems described herein. Separate data stores may becombined into a single data store, or a single data store may be splitinto two or more separate data stores.

Some of the functionality described herein may be implemented in thecontext of a client-server relationship. In this context, server devicesmay include suitable computing devices configured to provide informationand/or services described herein. Server devices may include anysuitable computing devices, such as dedicated server devices. Serverfunctionality provided by server devices may, in some cases, be providedby software (e.g., virtualized computing instances or applicationobjects) executing on a computing device that is not a dedicated serverdevice. The term “client” can be used to refer to a computing devicethat obtains information and/or accesses services provided by a serverover a communication link. However, the designation of a particulardevice as a client device does not necessarily require the presence of aserver. At various times, a single device may act as a server, a client,or both a server and a client, depending on context and configuration.Actual physical locations of clients and servers are not necessarilyimportant, but the locations can be described as “local” for a clientand “remote” for a server to illustrate a common usage scenario in whicha client is receiving information provided by a server at a remotelocation. Alternatively, a peer-to-peer arrangement, or other models,can be used.

FIG. 6 is a block diagram that illustrates aspects of an illustrativecomputing device 600 appropriate for use in accordance with embodimentsof the present disclosure. The description below is applicable toservers, personal computers, mobile phones, smart phones, tabletcomputers, embedded computing devices, and other currently available oryet-to-be-developed devices that may be used in accordance withembodiments of the present disclosure. Computing devices describedherein may be integrated with specialized hardware, such as microscopysystems, for obtaining images, or as stand-alone devices that obtainimages for analysis in some other way, such as by receiving imagesstored remotely in a cloud computing arrangement.

In its most basic configuration, the computing device 600 includes atleast one processor 602 and a system memory 604 connected by acommunication bus 606. Depending on the exact configuration and type ofdevice, the system memory 604 may be volatile or nonvolatile memory,such as read only memory (“ROM”), random access memory (“RAM”), EEPROM,flash memory, or other memory technology. Those of ordinary skill in theart and others will recognize that system memory 604 typically storesdata and/or program modules that are immediately accessible to and/orcurrently being operated on by the processor 602. In this regard, theprocessor 602 may serve as a computational center of the computingdevice 600 by supporting the execution of instructions.

As further illustrated in FIG. 6, the computing device 600 may include anetwork interface 610 comprising one or more components forcommunicating with other devices over a network. Embodiments of thepresent disclosure may access basic services that utilize the networkinterface 610 to perform communications using common network protocols.The network interface 610 may also include a wireless network interfaceconfigured to communicate via one or more wireless communicationprotocols, such as WiFi, 2G, 3G, 4G, LTE, WiMAX, Bluetooth, and/or thelike.

In the illustrative embodiment depicted in FIG. 6, the computing device600 also includes a storage medium 608. However, services may beaccessed using a computing device that does not include means forpersisting data to a local storage medium. Therefore, the storage medium608 depicted in FIG. 6 is optional. In any event, the storage medium 608may be volatile or nonvolatile, removable or nonremovable, implementedusing any technology capable of storing information such as, but notlimited to, a hard drive, solid state drive, CD-ROM, DVD, or other diskstorage, magnetic tape, magnetic disk storage, and/or the like.

As used herein, the term “computer-readable medium” includes volatileand nonvolatile and removable and nonremovable media implemented in anymethod or technology capable of storing information, such ascomputer-readable instructions, data structures, program modules, orother data. In this regard, the system memory 604 and storage medium 608depicted in FIG. 6 are examples of computer-readable media.

For ease of illustration and because it is not important for anunderstanding of the claimed subject matter, FIG. 6 does not show someof the typical components of many computing devices. In this regard, thecomputing device 600 may include input devices, such as a keyboard,keypad, mouse, trackball, microphone, video camera, touchpad,touchscreen, electronic pen, stylus, and/or the like. Such input devicesmay be coupled to the computing device 600 by wired or wirelessconnections including RF, infrared, serial, parallel, Bluetooth, USB, orother suitable connection protocols using wireless or physicalconnections.

In any of the described examples, input data can be captured by inputdevices and processed, transmitted, or stored (e.g., for futureprocessing). The processing may include encoding data streams, which canbe subsequently decoded for presentation by output devices. Media datacan be captured by multimedia input devices and stored by saving mediadata streams as files on a computer-readable storage medium (e.g., inmemory or persistent storage on a client device, server, administratordevice, or some other device). Input devices can be separate from andcommunicatively coupled to computing device 600 (e.g., a client device),or can be integral components of the computing device 600. In someembodiments, multiple input devices may be combined into a single,multifunction input device (e.g., a video camera with an integratedmicrophone). The computing device 600 may also include output devicessuch as a display, speakers, printer, etc. The output devices mayinclude video output devices such as a display or touchscreen. Theoutput devices also may include audio output devices such as externalspeakers or earphones. The output devices can be separate from andcommunicatively coupled to the computing device 600, or can be integralcomponents of the computing device 600. Input functionality and outputfunctionality may be integrated into the same input/output device (e.g.,a touchscreen). Any suitable input device, output device, or combinedinput/output device either currently known or developed in the futuremay be used with described systems.

In general, functionality of computing devices described herein may beimplemented in computing logic embodied in hardware or softwareinstructions, which can be written in a programming language, such as C,C++, COBOL, JAVA™, PHP, Perl, Python, Ruby, HTML, CSS, JavaScript,VBScript, ASPX, Microsoft .NET™ languages such as C #, and/or the like.Computing logic may be compiled into executable programs or written ininterpreted programming languages. Generally, functionality describedherein can be implemented as logic modules that can be duplicated toprovide greater processing capability, merged with other modules, ordivided into sub-modules. The computing logic can be stored in any typeof computer-readable medium (e.g., a non-transitory medium such as amemory or storage medium) or computer storage device and be stored onand executed by one or more general-purpose or special-purposeprocessors, thus creating a special-purpose computing device configuredto provide functionality described herein.

Extensions and Alternatives

Many alternatives to the systems and devices described herein arepossible. For example, individual modules or subsystems can be separatedinto additional modules or subsystems or combined into fewer modules orsubsystems. As another example, modules or subsystems can be omitted orsupplemented with other modules or subsystems. As another example,functions that are indicated as being performed by a particular device,module, or subsystem may instead be performed by one or more otherdevices, modules, or subsystems. Although some examples in the presentdisclosure include descriptions of devices comprising specific hardwarecomponents in specific arrangements, techniques and tools describedherein can be modified to accommodate different hardware components,combinations, or arrangements. Further, although some examples in thepresent disclosure include descriptions of specific usage scenarios,techniques and tools described herein can be modified to accommodatedifferent usage scenarios. Functionality that is described as beingimplemented in software can instead be implemented in hardware, or viceversa.

Many alternatives to the techniques described herein are possible. Forexample, processing stages in the various techniques can be separatedinto additional stages or combined into fewer stages. As anotherexample, processing stages in the various techniques can be omitted orsupplemented with other techniques or processing stages. As anotherexample, processing stages that are described as occurring in aparticular order can instead occur in a different order. As anotherexample, processing stages that are described as being performed in aseries of steps may instead be handled in a parallel fashion, withmultiple modules or software processes concurrently handling one or moreof the illustrated processing stages. As another example, processingstages that are indicated as being performed by a particular device ormodule may instead be performed by one or more other devices or modules.

The principles, representative embodiments, and modes of operation ofthe present disclosure have been described in the foregoing description.However, aspects of the present disclosure which are intended to beprotected are not to be construed as limited to the particularembodiments disclosed. Further, the embodiments described herein are tobe regarded as illustrative rather than restrictive. It will beappreciated that variations and changes may be made by others, andequivalents employed, without departing from the spirit of the presentdisclosure. Accordingly, it is expressly intended that all suchvariations, changes, and equivalents fall within the spirit and scope ofthe claimed subject matter.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. A computer-implementedmethod comprising: automatically converting a set of training images ofcells and related outcome metadata into a description document byextracting features from the training images that describe the cells andassociating the extracted features with the outcome metadata, whereinthe training images are obtained using bright-field microscopy; based onthe description document, automatically computing a decision model thatcan be used to predict outcomes of new cells, wherein the cells areoocytes or pronuclear embryos, and wherein the extracted featuresinclude one or more of the following: a boundary of a polar body,measured manually or with image analysis using an edge detectionalgorithm; an indication as to whether the polar body exists;identification of inner and outer edges of a zona pellucida;identification of perivitelline space; a measurement of smoothness ofthe edges of the zona pellucida; alignment of principal axes of thecytoplasm and the zona; total area of the polar body, the zonapellucida, or the perivitelline space; aspect ratio of the polar body,the zona pellucida, or the perivitelline space; convexity of a boundaryof the polar body, the zona pellucida, or the perivitelline space;average image intensity of the polar body, the zona pellucida, or theperivitelline space; standard deviation of the image intensity of thepolar body, the zona pellucida, or the perivitelline space; smoothnessof the polar body, the zona pellucida, or the perivitelline space; andsubsampled image intensity or smoothness for grid areas within the polarbody, the zona pellucida, or the perivitelline space.
 2. The method ofclaim 1, wherein the one or more outcomes include one or more of thefollowing: quality or reproductive potential of a new oocyte, whetherabnormalities in chromosome number are detected later in development,whether genetic disorders are detected later in development, andabnormal cell growth outcomes.
 3. The method of claim 2, wherein thequality or reproductive potential of the new oocyte is expressed as oneor more of: a probability that the oocyte will fertilize when exposed tohuman sperm, a probability that a resulting embryo will reach a givenstage of development, or a probability that the resulting embryo willresult in a live birth when transferred to a uterus.
 4. The method ofclaim 1 further comprising: automatically extracting features fromimages that describe new cells, wherein the images that describe the newcells are obtained using bright-field microscopy; predicting one or moreoutcomes for the new cells by applying the decision model to thefeatures extracted from the images that describe the new cells.
 5. Themethod of claim 4, wherein the features extracted from the images thatdescribe new cells correspond to features selected for inclusion in thedecision model.
 6. The method of claim 4, wherein the one or moreoutcomes include one or more of quality or reproductive potential for anew oocyte, whether abnormalities in chromosome number are detectedlater in development, whether genetic disorders are detected later indevelopment, and abnormal cell growth outcomes.
 7. The method of claim6, wherein the quality or reproductive potential of the new oocyte isexpressed as one or more of: a probability that the oocyte willfertilize when exposed to human sperm, a probability that a resultingembryo will reach a given stage of development, or a probability thatthe resulting embryo will result in a live birth when transferred to auterus.
 8. A computer-implemented method comprising: automaticallyconverting a set of training images of cells and related outcomemetadata into a description document by extracting features from thetraining images that describe the cells and associating the extractedfeatures with the outcome metadata, wherein the training images areobtained using bright-field microscopy; based on the descriptiondocument, automatically computing a decision model that can be used topredict outcomes of new cells, wherein the cells are oocytes orpronuclear embryos, and wherein the extracted features include one ormore of the following for a pronuclear embryo comprising pronuclei:identification of the boundaries of the pronuclei, measured manually orwith image analysis using an edge detection algorithm; area of thepronuclei, measured individually and/or relative to each other; aspectratio of the pronuclei, measured individually and/or relative to eachother; convexity of a boundary of one or more of the pronuclei; averageimage intensity of one or more of the pronuclei; standard deviation ofthe image intensity of one or more of the pronuclei; smoothness of oneor more of the pronuclei; and subsampled image intensity or smoothnessfor grid areas within one or more of the pronuclei.
 9. The method ofclaim 8, wherein the one or more outcomes include one or more of thefollowing: whether abnormalities in chromosome number are detected laterin development, whether genetic disorders are detected later indevelopment, and abnormal cell growth outcomes.
 10. The method of claim8 further comprising: automatically extracting features from images thatdescribe new cells, wherein the images that describe the new cells areobtained using bright-field microscopy; predicting one or more outcomesfor the new cells by applying the decision model to the featuresextracted from the images that describe the new cells.
 11. The method ofclaim 10, wherein the features extracted from the images that describenew cells correspond to features selected for inclusion in the decisionmodel.
 12. A computer-implemented method comprising: automaticallyextracting features that describe new cells from new cell images; andpredicting one or more outcomes for the new cells by applying a decisionmodel to the features extracted from the new cell images, wherein thedecision model is based on a description document that associatesoutcome metadata with features extracted from training images, whereinthe training images and the new cell images are obtained usingbright-field microscopy, wherein the new cells are oocytes or pronuclearembryos, and wherein the features extracted from the new cell images andthe features extracted from the training images include one or more ofthe following: a boundary of a polar body, measured manually or withimage analysis using an edge detection algorithm; an indication as towhether the polar body exists; identification of inner and outer edgesof a zona pellucida; identification of perivitelline space; ameasurement of smoothness of the edges of the zona pellucida; alignmentof principal axes of the cytoplasm and the zona; total area of the polarbody, the zona pellucida, or the perivitelline space; aspect ratio ofthe polar body, the zona pellucida, or the perivitelline space;convexity of the boundary of the polar body, the zona pellucida, or theperivitelline space; average image intensity of the polar body, the zonapellucida, or the perivitelline space; standard deviation of the imageintensity of the polar body, the zona pellucida, or the perivitellinespace; smoothness of the polar body, the zona pellucida, or theperivitelline space; and subsampled image intensity or smoothness forgrid areas within the polar body, the zona pellucida, or theperivitelline space.
 13. The method of claim 12, wherein the one or moreoutcomes include one or more of the following: quality or reproductivepotential of a new oocyte, whether abnormalities in chromosome numberare detected later in development, whether genetic disorders aredetected later in development, and abnormal cell growth outcomes. 14.The method of claim 13, wherein the quality or reproductive potential ofthe new oocyte is expressed as one or more of: a probability that theoocyte will fertilize when exposed to human sperm, a probability that aresulting embryo will reach a given stage of development, or aprobability that the resulting embryo will result in a live birth whentransferred to a uterus.
 15. A computer-implemented method comprising:automatically extracting features that describe new cells from new cellimages; and predicting one or more outcomes for the new cells byapplying a decision model to the features extracted from the new cellimages, wherein the decision model is based on a description documentthat associates outcome metadata with features extracted from trainingimages, wherein the training images and the new cell images are obtainedusing bright-field microscopy, wherein the new cells are oocytes orpronuclear embryos, and wherein the features extracted from the new cellimages and the features extracted from the training images include oneor more of the following for a pronuclear embryo comprising pronuclei:identification of the boundaries of the pronuclei, measured manually orwith image analysis using an edge detection algorithm; area of thepronuclei, measured individually and/or relative to each other; aspectratio of the pronuclei, measured individually and/or relative to eachother; convexity of a boundary of one or more of the pronuclei; averageimage intensity of one or more of the pronuclei; standard deviation ofthe image intensity of one or more of the pronuclei; smoothness of oneor more of the pronuclei; and subsampled image intensity or smoothnessfor grid areas within one or more of the pronuclei.
 16. The method ofclaim 15, wherein the one or more outcomes include one or more of thefollowing: whether abnormalities in chromosome number are detected laterin development, whether genetic disorders are detected later indevelopment, and abnormal cell growth outcomes.
 17. Acomputer-implemented method comprising: automatically extractingfeatures that describe new cells from new cell images; and predictingone or more outcomes for the new cells by applying a decision model tothe features extracted from the new cell images, wherein the decisionmodel is based on a description document that associates outcomemetadata with features extracted from training images, wherein thetraining images and the new cell images are obtained using bright-fieldmicroscopy, wherein the new cells are oocytes or pronuclear embryos, andwherein the one or more outcomes include one or more of the following:quality or reproductive potential for a new oocyte, whetherabnormalities in chromosome number are detected later in development,whether genetic disorders are detected later in development, andabnormal cell growth outcomes.
 18. The method of claim 17, wherein thequality or reproductive potential of the new oocyte is expressed as oneor more of: a probability that the oocyte will fertilize when exposed tohuman sperm, a probability that a resulting embryo will reach a givenstage of development, or a probability that the resulting embryo willresult in a live birth when transferred to a uterus.
 19. Anon-transitory computer-readable medium having stored thereoncomputer-executable instructions configured to cause one or morecomputing devices to perform steps comprising: automatically extractingfeatures that describe new cells from new cell images; and predictingone or more outcomes for the new cells by applying a decision model tothe features extracted from the new cell images, wherein the decisionmodel is based on a description document that associates outcomemetadata with features extracted from training images, wherein thetraining images and the new cell images are obtained using bright-fieldmicroscopy, wherein the new cells are oocytes or pronuclear embryos, andwherein the one or more outcomes include one or more of the following:quality or reproductive potential of a new oocyte, whether abnormalitiesin chromosome number are detected later in development, whether geneticdisorders are detected later in development, and abnormal cell growthoutcomes.
 20. The computer-readable medium of claim 19, wherein thefeatures extracted from the new cell images and the features extractedfrom the training images include one or more of the following: aboundary of a polar body, measured manually or with image analysis usingan edge detection algorithm; an indication as to whether the polar bodyexists; identification of inner and outer edges of a zona pellucida;identification of perivitelline space; a measurement of smoothness ofthe edges of the zona pellucida; alignment of principal axes of thecytoplasm and the zona; total area of the polar body, the zonapellucida, or the perivitelline space; aspect ratio of the polar body,the zona pellucida, or the perivitelline space; convexity of theboundary of the polar body, the zona pellucida, or the perivitellinespace; average image intensity of the polar body, the zona pellucida, orthe perivitelline space; standard deviation of the image intensity ofthe polar body, the zona pellucida, or the perivitelline space;smoothness of the polar body, the zona pellucida, or the perivitellinespace; and subsampled image intensity or smoothness for grid areaswithin the polar body, the zona pellucida, or the perivitelline space.21. The computer-readable medium of claim 19, wherein the featuresextracted from the new cell images and the features extracted from thetraining images include one or more of the following for a pronuclearembryo comprising pronuclei: identification of the boundaries of thepronuclei, measured manually or with image analysis using an edgedetection algorithm; area of the pronuclei, measured individually and/orrelative to each other; aspect ratio of the pronuclei, measuredindividually and/or relative to each other; convexity of a boundary ofone or more of the pronuclei; average image intensity of one or more ofthe pronuclei; standard deviation of the image intensity of one or moreof the pronuclei; smoothness of one or more of the pronuclei; andsubsampled image intensity or smoothness for grid areas within one ormore of the pronuclei.
 22. The computer-readable medium of claim 19,wherein the quality or reproductive potential of the new oocyte isexpressed as one or more of: a probability that the oocyte willfertilize when exposed to human sperm, a probability that a resultingembryo will reach a given stage of development, or a probability thatthe resulting embryo will result in a live birth when transferred to auterus.
 23. A system comprising one or more computing devices programmedto perform steps comprising: automatically extracting features thatdescribe new cells from new cell images; and predicting one or moreoutcomes for the new cells by applying a decision model to the featuresextracted from the new cell images, wherein the decision model is basedon a description document that associates outcome metadata with featuresextracted from training images, wherein the training images and the newcell images are obtained using bright-field microscopy, wherein the newcells are oocytes or pronuclear embryos, and wherein the one or moreoutcomes include one or more of the following: quality or reproductivepotential of a new oocyte, whether abnormalities in chromosome numberare detected later in development, whether genetic disorders aredetected later in development, and abnormal cell growth outcomes. 24.The system of claim 23 further comprising a microscopy system integratedwith the one or more computing devices.
 25. The system of claim 23,wherein the features extracted from the new cell images and the featuresextracted from the training images include one or more of the following:a boundary of a polar body, measured manually or with image analysisusing an edge detection algorithm; an indication as to whether the polarbody exists; identification of inner and outer edges of a zonapellucida; identification of perivitelline space; a measurement ofsmoothness of the edges of the zona pellucida; alignment of principalaxes of the cytoplasm and the zona; total area of the polar body, thezona pellucida, or the perivitelline space; aspect ratio of the polarbody, the zona pellucida, or the perivitelline space; convexity of theboundary of the polar body, the zona pellucida, or the perivitellinespace; average image intensity of the polar body, the zona pellucida, orthe perivitelline space; standard deviation of the image intensity ofthe polar body, the zona pellucida, or the perivitelline space;smoothness of the polar body, the zona pellucida, or the perivitellinespace; and subsampled image intensity or smoothness for grid areaswithin the polar body, the zona pellucida, or the perivitelline space.26. The system of claim 23, wherein the features extracted from the newcell images and the features extracted from the training images includeone or more of the following for a pronuclear embryo comprisingpronuclei: identification of the boundaries of the pronuclei, measuredmanually or with image analysis using an edge detection algorithm; areaof the pronuclei, measured individually and/or relative to each other;aspect ratio of the pronuclei, measured individually and/or relative toeach other; convexity of a boundary of one or more of the pronuclei;average image intensity of one or more of the pronuclei; standarddeviation of the image intensity of one or more of the pronuclei;smoothness of one or more of the pronuclei; and subsampled imageintensity or smoothness for grid areas within one or more of thepronuclei.
 27. The system of claim 23, wherein the quality orreproductive potential of the new oocyte is expressed as one or more of:a probability that the oocyte will fertilize when exposed to humansperm, a probability that a resulting embryo will reach a given stage ofdevelopment, or a probability that the resulting embryo will result in alive birth when transferred to a uterus.